Don't be misled: Three misconceptions about external validation of clinical prediction models.

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing three common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies.

HIPPO: HIstogram-based Pseudo-POtential for scoring protein-ssRNA fragment-based docking poses.

BACKGROUND: The RNA-Recognition motif (RRM) is a protein domain that binds single-stranded RNA (ssRNA) and is present in as much as 2% of the human genome. Despite this important role in biology, RRM-ssRNA interactions are very challenging to study on the structural level because of the remarkable flexibility of ssRNA. In the absence of atomic-level experimental data, the only method able to predict the 3D structure of protein-ssRNA complexes with any degree of accuracy is ssRNA'TTRACT, an ssRNA fragment-based docking approach using ATTRACT. However, since ATTRACT parameters are not ssRNA-specific and were determined in 2010, there is substantial opportunity for enhancement. RESULTS: Here we present HIPPO, a composite RRM-ssRNA scoring potential derived analytically from contact frequencies in near-native versus non-native docking models. HIPPO consists of a consensus of four distinct potentials, each extracted from a distinct reference pool of protein-trinucleotide docking decoys. To score a docking pose with one potential, for each pair of RNA-protein coarse-grained bead types, each contact is awarded or penalised according to the relative frequencies of this contact distance range among the correct and incorrect poses of the reference pool. Validated on a fragment-based docking benchmark of 57 experimentally solved RRM-ssRNA complexes, HIPPO achieved a threefold or higher enrichment for half of the fragments, versus only a quarter with the ATTRACT scoring function. In particular, HIPPO drastically improved the chance of very high enrichment (12-fold or higher), a scenario where the incremental modelling of entire ssRNA chains from fragments becomes viable. However, for the latter result, more research is needed to make it directly practically applicable. Regardless, our approach already improves upon the state of the art in RRM-ssRNA modelling and is in principle extendable to other types of protein-nucleic acid interactions.

Exploring a Structural Data Mining Approach to Design Linkers for Head-to-Tail Peptide Cyclization.

Peptides have recently regained interest as therapeutic candidates, but their development remains confronted with several limitations including low bioavailability. Backbone head-to-tail cyclization, i.e., setting a covalent peptide bond linking the last amino acid with the first one, is one effective strategy of peptide-based drug design to stabilize the conformation of bioactive peptides while preserving peptide properties in terms of low toxicity, binding affinity, target selectivity, and preventing enzymatic degradation. Starting from an active peptide, it usually requires the design of a linker of a few amino acids to make it possible to cyclize the peptide, possibly preserving the conformation of the initial peptide and not affecting its activity. However, very little is known about the sequence-structure relationship requirements of designing linkers for peptide cyclization in a rational manner. Recently, we have shown that large-scale data-mining of available protein structures can lead to the precise identification of protein loop conformations, even from remote structural classes. Here, we transpose this approach to linkers, allowing head-to-tail peptide cyclization. First we show that given a linker sequence and the conformation of the linear peptide, it is possible to accurately predict the cyclized peptide conformation. Second, and more importantly, we show that it seems possible to elaborate on the information inferred from protein structures to propose effective candidate linker sequences constrained by length and amino acid composition, providing the first framework for the rational design of head-to-tail cyclization linkers. Finally, we illustrate this for two peptides using a limited set of amino-acids likely not to interfere with peptide function. For a linear peptide derived from Nrf2, the peptide cyclized starting from the experimental structure showed a 26-fold increase in the binding affinity. For urotensin II, a peptide already cyclized by a disulfide bond that exerts a broad array of biological activities, we were able, starting from models of the structure, to design a head-to-tail cyclized peptide, the first synthesized bicyclic 14-residue long urotensin II analogue, showing a retention of in vitro activity. Although preliminary, our results strongly suggest that such an approach has strong potential for cyclic peptide-based drug design.

PEP-FOLD4: a pH-dependent force field for peptide structure prediction in aqueous solution.

Accurate and fast structure prediction of peptides of less 40 amino acids in aqueous solution has many biological applications, but their conformations are pH- and salt concentration-dependent. In this work, we present PEP-FOLD4 which goes one step beyond many machine-learning approaches, such as AlphaFold2, TrRosetta and RaptorX. Adding the Debye-Hueckel formalism for charged-charged side chain interactions to a Mie formalism for all intramolecular (backbone and side chain) interactions, PEP-FOLD4, based on a coarse-grained representation of the peptides, performs as well as machine-learning methods on well-structured peptides, but displays significant improvements for poly-charged peptides. PEP-FOLD4 is available at http://bioserv.rpbs.univ-paris-diderot.fr/services/PEP-FOLD4. This server is free and there is no login requirement.

ProtNAff: protein-bound Nucleic Acid filters and fragment libraries.

MOTIVATION: Atomistic models of nucleic acids (NA) fragments can be used to model the 3D structures of specific protein-NA interactions and address the problem of great NA flexibility, especially in their single-stranded regions. One way to obtain relevant NA fragments is to extract them from existing 3D structures corresponding to the targeted context (e.g. specific 2D structures, protein families, sequences) and to learn from them. Several databases exist for specific NA 3D motifs, especially in RNA, but none can handle the variety of possible contexts. RESULTS: This article presents protNAff (protein-bound Nucleic Acids filters and fragments), a new pipeline for the conception of searchable databases on the 2D and 3D structures of protein-bound NA, the selection of context-specific (regions of) NA structures by combinations of filters, and the creation of context-specific NA fragment libraries. The strength of this pipeline is its modularity, allowing users to adapt it to many specific modeling problems. As examples, the pipeline is applied to the quantitative analysis of (i) the sequence-specificity of trinucleotide conformations, (ii) the conformational diversity of RNA at several levels of resolution, (iii) the effect of protein binding on RNA local conformations and (iv) the protein-binding propensity of RNA hairpin loops of various lengths. AVAILABILITY AND IMPLEMENTATION: The source code is freely available for download at URL https://github.com/isaureCdB/protNAff. The database and the trinucleotide fragment library are downloadable at URL https://zenodo.org/record/6483823#.YmbVhFxByV4. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

A semi-supervised decision support system to facilitate antibiotic stewardship for urinary tract infections.

Urinary Tract Infections (UTIs) are among the most frequently occurring infections in the hospital. Urinalysis and urine culture are the main tools used for diagnosis. Whereas urinalysis is sufficiently sensitive for detecting UTI, it has a relatively low specificity, leading to unnecessary treatment with antibiotics and the risk of increasing antibiotic resistance. We performed an evaluation of the current diagnostic process with an expert-based label for UTI as outcome, retrospectively established using data from the Electronic Health Records. We found that the combination of urinalysis results with the Gram stain and other readily available parameters can be used effectively for predicting UTI. Based on the obtained information, we engineered a clinical decision support system (CDSS) using the reliable semi-supervised ensemble learning (RESSEL) method, and found it to be more accurate than urinalysis or the urine culture for prediction of UTI. The CDSS provides clinicians with this prediction within hours of ordering a culture and thereby enables them to hold off on prematurely prescribing antibiotics for UTI while awaiting the culture results.

SeamDock: An Interactive and Collaborative Online Docking Resource to Assist Small Compound Molecular Docking.

In silico assessment of protein receptor interactions with small ligands is now part of the standard pipeline for drug discovery, and numerous tools and protocols have been developed for this purpose. With the SeamDock web server, we propose a new approach to facilitate access to small molecule docking for nonspecialists, including students. The SeamDock online service integrates different docking tools in a common framework that allows ligand global and/or local docking and a hierarchical approach combining the two for easy interaction site identification. This service does not require advanced computer knowledge, and it works without the installation of any programs with the exception of a common web browser. The use of the Seamless framework linking the RPBS calculation server to the user's browser allows the user to navigate smoothly and interactively on the SeamDock web page. A major effort has been put into the 3D visualization of ligand, receptor, and docking poses and their interactions with the receptor. The advanced visualization features combined with the seamless library allow a user to share with an unlimited number of collaborators, a docking session, and its full visualization states. As a result, SeamDock can be seen as a free, simple, didactic, evolving online docking resource best suited for education and training.

The search of sequence variants using a constrained protein evolution simulation approach.

Protein engineering or candidate therapeutic peptide optimization are processes in which the identification of relevant sequence variants is critical. Starting from one amino-acid sequence, the choice of the substitutions must meet the objective of not disrupting the structure of the protein, not impacting the main functional properties of the starting entity, while also meeting the condition to enhance some expected property such as thermal stability, resistance to degradation, … Here, we introduce a new approach of sequence evolution that focuses on the objective of not disrupting the structure of the initial protein by embedding a point to point control on the preservation of the local structure at each position in the sequence. For 6 mini-proteins, we find that, starting from a single sequence, our simple approach intrinsically contains information about site-specific rate heterogeneity of substitution, and that it is able to reproduce sequence diversity as can be observed in the sequences available in the Uniref repository. We show that our approach is able to provide information about positions not to substitute and about substitutions not to perform at a given position to maintain structure integrity. Overall, our results demonstrate that point to point preservation of the local structure along a sequence is an important determinant of sequence evolution.

Coarse-grained and atomic resolution biomolecular docking with the ATTRACT approach.

The ATTRACT protein-protein docking program has been employed to predict protein-protein complex structures in CAPRI rounds 38-45. For 11 out of 16 targets acceptable or better quality solutions have been submitted (~70%). It includes also several cases of peptide-protein docking and the successful prediction of the geometry of carbohydrate-protein interactions. The option of combining rigid body minimization and simultaneous optimization in collective degrees of freedom based on elastic network modes was employed and systematically evaluated. Application to a large benchmark set indicates a modest improvement in docking performance compared to rigid docking. Possible further improvements of the docking approach in particular at the scoring and the flexible refinement steps are discussed.

DaReUS-Loop: a web server to model multiple loops in homology models.

Loop regions in protein structures often have crucial roles, and they are much more variable in sequence and structure than other regions. In homology modeling, this leads to larger deviations from the homologous templates, and loop modeling of homology models remains an open problem. To address this issue, we have previously developed the DaReUS-Loop protocol, leading to significant improvement over existing methods. Here, a DaReUS-Loop web server is presented, providing an automated platform for modeling or remodeling loops in the context of homology models. This is the first web server accepting a protein with up to 20 loop regions, and modeling them all in parallel. It also provides a prediction confidence level that corresponds to the expected accuracy of the loops. DaReUS-Loop facilitates the analysis of the results through its interactive graphical interface and is freely available at http://bioserv.rpbs.univ-paris-diderot.fr/services/DaReUS-Loop/.

Higher-level cognitive functions in Dutch elite and sub-elite table tennis players.

This study aimed to investigate the higher-level cognitive functions (i.e. metacognition and executive functions) of Dutch competitive table tennis players to better understand its relevance for performance in this fast and complex sport. Thirty elite (age 16 ± 4) and thirty age and sex-matched sub-elite peers (age 16 ± 5) were assessed on metacognition and executive functions (working memory, inhibitory control, cognitive flexibility) using D-KEFS tests. Compared to norm scores, both the Dutch competitive elite and sub-elite table tennis players scored above average on all tests (p < 0.05). MANOVA showed a main effect for performance level (elites outscored sub-elites; p < 0.05). T-tests revealed that elite players make less mistakes on tests for inhibitory control (CWI-3: 0.9 ± 0.9; CWI-4: 1.1 ± 1.2) than sub-elite players (CWI-3: 1.8 ± 1.1; CWI-4: 2.6 ± 1.5) (p < 0.05). When controlling for training hours in a MANCOVA, no significant main effect of performance level remained (p > 0.05). In conclusion, Dutch elite and sub-elite table tennis players are characterized by above-average scores on higher-level cognitive functions compared to norm scores. A relation with performance level has been shown, which may be explained by the greater exposure to table tennis for elite compared to sub-elite players. However, longitudinal research is needed to indicate the direction of this association.

DaReUS-Loop: accurate loop modeling using fragments from remote or unrelated proteins.

Despite efforts during the past decades, loop modeling remains a difficult part of protein structure modeling. Several approaches have been developed in the framework of crystal structures. However, for homology models, the modeling of loops is still far from being solved. We propose DaReUS-Loop, a data-based approach that identifies loop candidates mining the complete set of experimental structures available in the Protein Data Bank. Candidate filtering relies on local conformation profile-profile comparison, together with physico-chemical scoring. Applied to three different template-based test sets, DaReUS-Loop shows significant increase in the number of high-accuracy loops, and significant enhancement for modeling long loops. A special advantage is that our method proposes a prediction confidence score that correlates well with the expected accuracy of the loops. Strikingly, over 50% of successful loop models are derived from unrelated proteins, indicating that fragments under similar constraints tend to adopt similar structure, beyond mere homology.

Fast and accurate grid representations for atom-based docking with partner flexibility.

Macromolecular docking methods can broadly be divided into geometric and atom-based methods. Geometric methods use fast algorithms that operate on simplified, grid-like molecular representations, while atom-based methods are more realistic and flexible, but far less efficient. Here, a hybrid approach of grid-based and atom-based docking is presented, combining precalculated grid potentials with neighbor lists for fast and accurate calculation of atom-based intermolecular energies and forces. The grid representation is compatible with simultaneous multibody docking and can tolerate considerable protein flexibility. When implemented in our docking method ATTRACT, grid-based docking was found to be ∼35x faster. With the OPLSX forcefield instead of the ATTRACT coarse-grained forcefield, the average speed improvement was >100x. Grid-based representations may allow atom-based docking methods to explore large conformational spaces with many degrees of freedom, such as multiple macromolecules including flexibility. This increases the domain of biological problems to which docking methods can be applied. © 2017 Wiley Periodicals, Inc.

The pepATTRACT web server for blind, large-scale peptide-protein docking.

Peptide-protein interactions are ubiquitous in the cell and form an important part of the interactome. Computational docking methods can complement experimental characterization of these complexes, but current protocols are not applicable on the proteome scale. pepATTRACT is a novel docking protocol that is fully blind, i.e. it does not require any information about the binding site. In various stages of its development, pepATTRACT has participated in CAPRI, making successful predictions for five out of seven protein-peptide targets. Its performance is similar or better than state-of-the-art local docking protocols that do require binding site information. Here we present a novel web server that carries out the rigid-body stage of pepATTRACT. On the peptiDB benchmark, the web server generates a correct model in the top 50 in 34% of the cases. Compared to the full pepATTRACT protocol, this leads to some loss of performance, but the computation time is reduced from ∼18 h to ∼10 min. Combined with the fact that it is fully blind, this makes the web server well-suited for large-scale in silico protein-peptide docking experiments. The rigid-body pepATTRACT server is freely available at http://bioserv.rpbs.univ-paris-diderot.fr/services/pepATTRACT.

Rapid Design of Knowledge-Based Scoring Potentials for Enrichment of Near-Native Geometries in Protein-Protein Docking.

Protein-protein docking protocols aim to predict the structures of protein-protein complexes based on the structure of individual partners. Docking protocols usually include several steps of sampling, clustering, refinement and re-scoring. The scoring step is one of the bottlenecks in the performance of many state-of-the-art protocols. The performance of scoring functions depends on the quality of the generated structures and its coupling to the sampling algorithm. A tool kit, GRADSCOPT (GRid Accelerated Directly SCoring OPTimizing), was designed to allow rapid development and optimization of different knowledge-based scoring potentials for specific objectives in protein-protein docking. Different atomistic and coarse-grained potentials can be created by a grid-accelerated directly scoring dependent Monte-Carlo annealing or by a linear regression optimization. We demonstrate that the scoring functions generated by our approach are similar to or even outperform state-of-the-art scoring functions for predicting near-native solutions. Of additional importance, we find that potentials specifically trained to identify the native bound complex perform rather poorly on identifying acceptable or medium quality (near-native) solutions. In contrast, atomistic long-range contact potentials can increase the average fraction of near-native poses by up to a factor 2.5 in the best scored 1% decoys (compared to existing scoring), emphasizing the need of specific docking potentials for different steps in the docking protocol.

Investigating the reliability and validity of the Dutch versions of the illness management and recovery scales among clients with mental disorders.

BACKGROUND: The Illness Management and Recovery scales (IMRS) can measure the progress of clients' illness self-management and recovery. Previous studies have examined the psychometric properties of the IMRS. AIMS: This study examined the reliability and validity of the Dutch version of the IMRS. METHOD: Clients (n = 111) and clinicians (n = 40) completed the client and clinician versions of the IMRS, respectively. The scales were administered again 2 weeks later to assess stability over time. Validity was assessed with the Utrecht Coping List (UCL), Dutch Empowerment Scale (DES), and Brief Symptom Inventory (BSI). RESULTS: The client and clinician versions of the IMRS had moderate internal reliability, with α = 0.69 and 0.71, respectively. The scales showed strong test-retest reliability, r = 0.79, for the client version and r = 0.86 for the clinician version. Correlations between client and clinician versions ranged from r = 0.37 to 0.69 for the total and subscales. We also found relationships in expected directions between the client IMRS and UCL, DES and BSI, which supports validity of the Dutch version of the IMRS. CONCLUSIONS: The Dutch version of the IMRS demonstrated good reliability and validity. The IMRS could be useful for Dutch-speaking programs interested in evaluating client progress on illness self-management and recovery.

Protein-protein and peptide-protein docking and refinement using ATTRACT in CAPRI.

The ATTRACT coarse-grained docking approach in combination with various types of atomistic, flexible refinement methods has been applied to predict protein-protein and peptide-protein complexes in CAPRI rounds 28-36. For a large fraction of CAPRI targets (12 out of 18), at least one model of acceptable or better quality was generated, corresponding to a success rate of 67%. In particular, for several peptide-protein complexes excellent predictions were achieved. In several cases, a combination of template-based modeling and extensive molecular dynamics-based refinement yielded medium and even high quality solutions. In one particularly challenging case, the structure of an ubiquitylation enzyme bound to the nucleosome was correctly predicted as a set of acceptable quality solutions. Based on the experience with the CAPRI targets, new interface refinement approaches and methods for ab-initio peptide-protein docking have been developed. Failures and possible improvements of the docking method with respect to scoring and protein flexibility will also be discussed. Proteins 2017; 85:391-398. © 2016 Wiley Periodicals, Inc.

SAXS Data Alone can Generate High-Quality Models of Protein-Protein Complexes.

Modeling driven by small-angle X-ray scattering (SAXS) combines low-resolution data with computational modeling to predict the structure of biomolecular assemblies. A new protocol, ATTRACT-SAXS, has been developed and tested on a large protein-protein docking benchmark with simulated SAXS data. For 88% of cases, high-quality solutions were generated using SAXS data alone without a physiochemical force field (interface-RMSD ≦ 2 Å or ligand-RMSD ≦ 5 Å; and more than 30% native contacts). ATTRACT-SAXS gave significant improvements compared with previous approaches that filter by SAXS a posteriori. When combining SAXS and interface properties for scoring, the protocol placed high-quality models in 70% of cases among the top-ranked 100 clusters. ATTRACT-SAXS also gave good results when tested on experimental data if the native complex structure was compatible with the SAXS profile. Our results show that, in principle, SAXS on its own can contain enough information to generate high-quality models of protein-protein complexes.

Fragment-based modelling of single stranded RNA bound to RNA recognition motif containing proteins.

Protein-RNA complexes are important for many biological processes. However, structural modeling of such complexes is hampered by the high flexibility of RNA. Particularly challenging is the docking of single-stranded RNA (ssRNA). We have developed a fragment-based approach to model the structure of ssRNA bound to a protein, based on only the protein structure, the RNA sequence and conserved contacts. The conformational diversity of each RNA fragment is sampled by an exhaustive library of trinucleotides extracted from all known experimental protein-RNA complexes. The method was applied to ssRNA with up to 12 nucleotides which bind to dimers of the RNA recognition motifs (RRMs), a highly abundant eukaryotic RNA-binding domain. The fragment based docking allows a precise de novo atomic modeling of protein-bound ssRNA chains. On a benchmark of seven experimental ssRNA-RRM complexes, near-native models (with a mean heavy-atom deviation of <3 Å from experiment) were generated for six out of seven bound RNA chains, and even more precise models (deviation < 2 Å) were obtained for five out of seven cases, a significant improvement compared to the state of the art. The method is not restricted to RRMs but was also successfully applied to Pumilio RNA binding proteins.

Recovery of motor imagery ability in the first year after stroke.

The aim of this study was to examine the level and recovery of motor imagery ability (MIA) in the first year after stroke and whether the recovery of MIA is related to that of arm/hand function. Twenty-three patients with diminished arm/hand function were included. The accuracy score on the hand laterality judgment task was used to assess MIA and the Fugl-Meyer Assessment was used to evaluate the recovery of arm/hand function. The patients were assessed 3, 6, 16, 26, and 52 weeks after stroke. In the first year after stroke, the percentage of patients with moderate to good MIA improved from 78% after 3 weeks to 94% after 1 year. The recovery of MIA took place in the first 6 weeks after stroke. No correlation was found between the recovery of MIA and arm/hand function, despite the fact that the greatest improvement in both occurred in the first 6 weeks.